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The School Improvement Specialist Project prepared seven modules. School improvement 
specialists, as defined by the Appalachia Educational Laboratory at Edvantia, are change agents 
who work with schools to help them improve in the following areas so as to increase student 
achievement. These modules are intended to provide training materials for educators seeking 
professional development to prepare them for a new level of work. 

Module 1 — Shared Leadership 

Module 2 — Learning Culture 

Module 3 — School-Lamily-Community Connections 

Module 4 — Effective Teaching 

Module 5 — Shared Goals for Learning 

Module 6 — Aligned and Balanced Curriculum 

Module 7 — Purposeful Student Assessment 

Each module has three sections: 

1. Standards: Each set of content standards and performance indicators helps school 
improvement specialists assess their skills and knowledge related to each topic. The 
rubric format provides both a measurement for self-assessment and goals for self- 
improvement. 

2. Improving Schools: These briefs provide research- and practice-based information to 
help school improvement specialists consider how they might address strengths and 
weaknesses in the schools where they work. The information contained in the briefs is 
often appropriate for sharing with teachers and principals; each includes information 
about strategies and practices that can be implemented in schools, resources to be 
consulted for more information, tools for facilitating thinking about and working on 
school issues, and real-life stories from school improvement specialists who offer 
their advice and experiences. 

3. Literature Review: The reviews of research literature summarize the best available 
information about the topic of each module. They can be used by school improvement 
specialists to expand their knowledge base and shared with school staffs as part of 
professional development activities. 




Purposeful Assessment 

Content Standards and Performance Indicators for School Improvement Specialists 

Self-Assessment Tool 

Purposeful Assessment: This matrix measures the extent to whieh a sehool improvement speeialist has the knowledge and skills to assist a sehool in developing and sustaining 
purposeful assessment as re fleeted by the following eharaeteristies: (1) interpreting and using data, (2) understanding and use of best assessment praetiees, (3) knowledge of 
student assessment, (4) artieulation of assessment for learning, and (5) knowledge of mandates and statutes regarding assessment. 



Knowledge or Skill 


Advanced 


Proficient 


Basic 


Novice 


1 . Interpreting and using 
data 


The sehool improvement speeialist 

a. helps institutionalize the praetiee among 
the entire sehool eommunity of using 
multiple sourees of student performanee 
data, as well as nonaeademie data, to 
analyze the eurrieulum or instruetional 
program 

b. teaehes faeulty to disaggregate data 
extensively 

e. faeilitates strategizing sessions for using 
the data to set goals for student aehieve- 
ment 

d. faeilitates sehool personnel in the use of 
data to monitor and adjust eurrieulum 
and instruetion for the purpose of 
eontinuous improvement 

e. faeilitates ongoing professional dialogue 
around student data and its relation to 
student aehievement 

f faeilitates diseussion around resourees 
needed to reaeh goals; models ways to 
identify, evaluate, and obtain those re- 
sourees 


The sehool improvement speeialist 

a. teaehes faeulty and administration to 
interpret multiple sourees of sehool data, 
ineluding both aeademie and non- 
aeademie measures 

b. works with faeulty to disaggregate data 

e. teaehes faeulty how to use assessment 

results to set goals for student aehieve- 
ment 

d. faeilitates diseussion of strengths and 
needs, and suggests ways to improve 
eurrieulum or instruetion 

e. faeilitates diseussion of data analysis 
several times throughout the sehool year 

f models ways to identify, evaluate, and 
obtain resourees for sehools to use in 
reaehing their goals 


The sehool improvement speeialist 

a. interprets multiple sourees of student 
performanee data and presents summa- 
ries to the entire sehool staff 

b. eompletes an extensive disaggregation of 
data and shares this information with the 
entire sehool staff 

e. suggests goals for student aehievement 
based on data 

d. shares strengths and needs based on data 
and suggests ways to improve eurrieulum 
or instruetion 

e. examines data several times throughout 
the sehool year, presenting summaries to 
the sehool staff 

f identifies, evaluates, and obtains re- 
sourees for sehools to use in reaehing 
their goals 


The sehool improvement speeialist 

a. interprets sehool data from standardized 
assessments and presents summaries to 
the sehool administration 

b. eompletes a limited disaggregation of 
data and shares with the administration 

e. shares strengths and needs identified by 
the data 

d. examines sehool data at the beginning of 
the sehool year and at the end of the 
sehool year 

e. identifies some resourees for sehools to 
use in addressing needs 
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Knowledge or Skill 


Advanced 


Proficient 


Basic 


Novice 


2. Understanding and use 
of best assessment 
praetiees 


The sehool improvement speeialist 

a. has a deep understanding of researeh 
eonneeting high-quality student assess- 
ment to student aehievement and strate- 
gieally shares this information with the 
sehool eommunity 

b. has extensive knowledge of multiple 
assessment types and eonduets ongoing 
dialogue with faeulty about ways to 
mateh assessments with eontent 

e. demonstrates knowledge of a wide va- 
riety of formative assessments and 
teaehes faeulty to use these 

d. has a deep understanding of best-praetiee 
methods of elassroom assessment and 
eonsistently shares and demonstrates 
these methods with faeulty in an 
organized, strategie manner 

e. faeilitates teaehers in development and 
eommon grading of benehmark assess- 
ments related to speeifie standards; faeili- 
tates dialogue about the meaning of the 
results and the desired next steps 

f thoroughly understands several eognitive 
models and faeilitates various opportuni- 
ties for teaehers to learn to ineorporate 
these into elassroom assessment praetiees 

g. eonsistently models ways to provide 
timely and appropriate feedbaek to 
students and teaehers on their progress 


The sehool improvement speeialist 

a. possesses a broad understanding of 
researeh eonneeting high-quality student 
assessment to student aehievement and 
explains this eonneetion to the sehool 
eommunity 

b. knows multiple assessment types and 
eneourages faeulty to vary assessment 
strategies 

e. models a variety of different formative 
assessments to teaehers 

d. understands best-praetiee methods for 
elassroom assessments and shares these 
methods with faeulty in an organized, 
strategie manner 

e. models and demonstrates development 
and eommon grading of benehmark 
assessments related to speeifie standards 

f understands and artieulates several 
eognitive models and demonstrates ways 
to ineorporate these levels of eognition 
into everyday assessment praetiees 

g. models and demonstrates methods for 
providing students with timely and 
appropriate feedbaek on their progress 


The sehool improvement speeialist 

a. understands the researeh eonneeting 
high-quality student assessment to 
student aehievement and shares this 
researeh with sehool administration 

b. ean identify different kinds of assess- 
ments for faeulty and staff, but eontinues 
to foeus primarily on traditional 
methodologies 

e. eneourages faeulty to use frequent 
short-term formative assessments 

d. knows some best-praetiee methods of 
elassroom assessments and shares these 
methods with teaehers, as requested 

e. eneourages the use of benehmark testing 
at set intervals throughout the sehool 
year, using the test data to modify 
instruetion 

f understands and artieulates at least one 
model of eognition and eneourages 
teaehers to use assessments requiring 
higher eognitive levels of response 

g. shares some methods for providing 
students with timely feedbaek on their 
progress 


The sehool improvement speeialist 

a. has minimal knowledge of researeh 
eonneeting high-quality student assess- 
ment to student aehievement 

b. has some knowledge of assessment 
praetiees, but foeuses primarily on 
summative assessments and traditional 
methodologies 

e. ean loeate sample assessment items from 
state and loeal sourees and from the 
Internet 

d. has a rudimentary knowledge about ways 
to ereate elassroom assessments 

e. is knowledgeable about benehmark 
assessments, but reeommends only basie 
and easily loeated sourees 

f understands and ean artieulate at least 
one model of eognition, sueh as Bloom’s 
taxonomy 

g. eneourages teaehers to provide students 
with timely feedbaek on their progress 
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Knowledge or Skill 


Advanced 


Proficient 


Basic 


Novice 


3. Knowledge of student 
self-assessment 


The sehool improvement speeialist 

a. has a detailed eomprehension of the 
value of student self-assessment and 
elearly artieulates this value to the entire 
sehool eommunity 

b. models and shares different self- 
assessment methods with the sehool 
eommunity 

e. demonstrates extensive self-refleetion 
methods, ineluding both aeademie and 
behavioral indieators, and faeilitates 
dialogue about the power of sueh instru- 
ments in inereasing student aehievement 

d. demonstrates motivational strategies that 
enable students to take responsibility for 
their own work and eonsistently 
faeilitates faeulty refleetion on and 
development of further strategies 


The sehool improvement speeialist 

a. ean elearly artieulate the value of student 
self-assessment 

b. demonstrates use of different self- 
assessment methods 

e. models and demonstrates several self- 
refleetion methods, ineorporating both 
aeademie and behavioral indieators, with 
teaehers and students 

d. demonstrates motivational strategies that 
enable students to take responsibility for 
their own work 


The sehool improvement speeialist 

a. understands the value of student self 
assessment 

b. eneourages faeulty to use some student 
self-assessment methods 

e. shares self-refleetion methods sueh as 
learning logs with teaehers and students 

d. eneourages teaehers to use motivational 
strategies that enable students to take 
responsibility for their own work 


The sehool improvement speeialist 

a. has an appreeiation for the value of 
student self-assessment but has limited 
knowledge in ways to aehieve it, 
eoneentrating instead on teaeher-eentered 
assessment strategies 

b. uses limited refleetion teehniques for 
self-assessment with teaehers or with 
students 
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Knowledge or Skill 


Advanced 


Proficient 


Basic 


Novice 


4. Artieulation of 
assessment for 
learning 


The sehool improvement speeialist 

a. has a deep understanding of the eoneept 
of assessment for learning and mentors 
sehool leaders and others in ereating a 
eulture that sustains this philosophy 

b. faeilitates the sehool eommunity in un- 
derstanding assessment as a key 
eomponent of instruetion 

e. faeilitates faeulty in understanding and 
using protoeols for examining student 
work and leads refleetive dialogue in 
eonneeting assessment to instruetion 

d. models and demonstrates an in-depth use 
of student-eentered assessment, sueh as 
student involvement in developing units 
of study and assessment strategies, along 
with ongoing eolleetion of data on stu- 
dent learning; engages the sehool eom- 
munity in developing and diseussing 
student-eentered assessment strategies 

e. has a deep understanding of the barriers 
to ehanging assessment praetiees; works 
in groups and one-on-one to help 
overeome these barriers through 
information and eonsensus building 


The sehool improvement speeialist 

a. understands the eoneept of assessment 
for learning and leads diseussions within 
the sehool eommunity about ereating a 
eulture of sueh assessment 

b. demonstrates prineiples and methods of 
assessing for learning to the entire sehool 
eommunity in a strategie manner 

e. faeilitates faeulty in understanding and 
using protoeols for examining student 
work and ereates opportunities for 
eollaborative diseussion of student work 

d. demonstrates the use of student-eentered 
assessment, sueh as student feedbaek on 
elassroom experienees and student 
refleeetive logs on their learning styles 

e. has knowledge of barriers to ehanging 
assessment and works with the sehool to 
overeome these barriers 


The sehool improvement speeialist 

a. understands the eoneept of assessment 
for learning and shares this eoneept with 
the entire sehool eommunity 

b. ereates and shares assessments for learn- 
ing with faeulty 

e. teaehes faeulty one or more strategies for 
examining student work and eneourages 
this praetiee 

d. artieulates the eoneept of student- 
eentered assessment to faeulty 

e. has a basie understanding of the barriers 
to ehanging assessment praetiee 


The sehool improvement speeialist 

a. understands the eoneept of assessment 
for learning on a limited basis and shares 
this eoneept with administrators and 
some teaehers 

b. loeates examples of assessments that 
demonstrate assessment for learning and 
shares them with the faeulty 

e. eneourages faeulty to examine student 
work eollaboratively 

d. eneourages faeulty to design assessments 
prior to instruetion and to get feedbaek 
from students 
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Knowledge or Skill 


Advanced 


Proficient 


Basic 


Novice 


5. Knowledge of 

mandates and statutes 
regarding assessment 


The sehool improvement speeialist 

a. has extensive knowledge of federal, state, 
and loeal regulations and standards 
eoneeming student assessment 

b. ean elearly artieulate the rationale and 
history behind the various regulations 
and standards 

e. faeilitates the sehool eommunity in 
eonversations toward understanding and 
respeet for the regulations and standards 

d. engages in refleetive dialogue with a 
professional network to understand, 
elarify, and possibly refine regulations 
and standards 


The sehool improvement speeialist 

a. has extensive knowledge and understand- 
ing of federal, state, and loeal regulations 
and standards eoneeming student 
assessment 

b. ean artieulate some of the rationale 
behind the regulations and standards 

e. works with the sehool eommunity to 
explain and elarify regulations and stan- 
dards 


The sehool improvement speeialist 

a. has knowledge of federal, state, and loeal 
regulations and standards eoneeming 
student assessment 

b. explains and elarifies regulations and 
standards to administration and faeulty 


The sehool improvement speeialist 

a. has knowledge of federal and state 
regulations and standards eoneeming 
student assessment, but may have limited 
understanding of loeal regulations 

b. explains regulations to sehool 
administration 
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Assessment for Learning 

As long as we have had school, we have had testing, because educators need to 
know how well their students are performing at the tasks they are learning. For years, 
teachers have instructed, tested, and then issued student grades after the instruction was 
complete. This mode of testing represents assessment o/learning. Summative 
assessments measure mastery and represent accountability for the teacher and the student. 
They are a way to determine the competence of any given student or group of students. 

While assessment of learning is extremely important, it gives only part of the 
information needed to modify instruction and increase student learning. Students need to 
see that the purpose of assessment is more than telling them whether they succeeded or 
failed. Assessment can be a tool for navigating the voyage from not knowing to 
understanding. Formative assessment is also a tool for refining curriculum — the plan for 
instruction. 

Formative assessment can help teachers and students make better judgments about 
what to do next. It asks teachers and students to become more explicit about the scope 
and sequence of instructional situations, to arrive at valid and reliable judgments about 
situations, and to make decisions for re-teaching students, changing instructional design, 
or improving curriculum design. By making formative assessment part of instructional 
design and curriculum development, districts can help teachers bring subconscious 
thoughts about situations, judgments, and next steps to the level where consensus can be 
reached about the best way to proceed (Black & Wiliam, October 1998). 

British researchers Paul Black and Dylan Wiliam (March 1998) conducted an 
extensive review of the research on assessment for the previous nine years. The more 
than 250 journal articles and books published about the research reveal that using 
formative assessments in classrooms can significantly increase student learning as 
measured by end-of-year standardized assessments. These findings also show that the 
increase is greater with low- achieving students than with other groups. 

Although research shows the value of formative classroom assessments, it also 
reveals some serious problems with the way formative assessments are used. Some tests 
encourage rote memorization. Too often, the primary purpose of assessments is to assign 
grades, which can encourage student competition rather than improvement. According to 
Black and Wiliam (October 1998), formative assessments are those used by teachers to 
revise their lessons to meet student needs. 

This view of formative assessment appears in the work of Rick Stiggins and 
colleagues (2004) and Jackie Walsh and Beth Sattes (2005). Educators are addressing the 
value of formative assessment and discovering the best ways to implement it in the 
classroom. The use of formative assessment to provide consistent, specific feedback to 
teachers and students is termed assessment for learning. The educators mentioned above 
identify strategies that can be used in assessment for learning. For example, effective 
questioning; When teachers provide “wait time” of 3 to 5 seconds after posing a question 
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and after a student answers, the quality of student response increases dramatically 
(Barnette et al., 1995; Hunkins, 1995; Rowe, 1986; Tobin, 1987). 

Other strategies for improving student performance include writing comments on 
student work. Comments show strengths and areas to improve, and help students to 
complete and refine their work. Likewise, when students have a clear rubric or scoring 
guide, they can effectively score their own and others’ work. Such collaborative work 
among students can also increase learning and achievement. 

In assessment for learning, students receive specific feedback about their own 
performance; they are not compared to other students. This feedback focuses on specific 
learning objectives and occurs throughout the learning period. Learning and assessment 
are integrally related one to another. The feedback helps students know where they need 
to increase their knowledge or skill, it helps teachers know what to emphasize in their 
lessons, and it demonstrates progress over time. With assessment for learning, students 
frequently engage in peer assessment or self-assessment. They are often able to lead 
conferences about their progress with their teachers and their parents. 

A challenge for educators is to consider and embrace the concept of assessment 
for learning. We need to ask ourselves some tough questions. Do students have 
assessments that relate to real life and that require multiple tasks of higher order skills? 
Do we plan assessment as part of the lesson? Do we revise either or both if the 
assessment results tell us students do not understand the content? Do we provide scoring 
guides in the form of rubrics so students know what to aim for? Do we give specific 
feedback on performance so students can set their learning goals and know where they 
are going? 

We do not want to abandon the concept of assessment of learning but we certainly 
want to balance it with a strong assessment for learning. 
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A New Look at Bloom’s Taxonomy 

Today there is great emphasis on designing instruction that pushes students to go 
beyond simple recall of facts and develop their abilities to think analytically, write 
critically, and use knowledge to create something new. As we push students toward 
higher levels in the classroom, we should also help teachers use assessments that are at 
higher cognitive levels. 

Bloom’s Taxonomy was created in 1956 as a tool for differentiating among levels 
of thinking, and most teachers are acquainted with it. In it, Benjamin Bloom defined six 
cognitive levels: knowledge, comprehension, application, analysis, synthesis, and 
evaluation. In 2001, Lorin Anderson and David Krathwohl published a book that presents 
the results of many years of collaboration among psychologists, curriculum experts, and 
researchers who contributed to refining and elaborating the original Bloom’s Taxonomy. 

This revised version of Bloom’s Taxonomy has several features useful for 
designing and assessing classroom instruction, and long-time teachers may appreciate 
making its acquaintance. First, the terminology is slightly different and a bit easier to 
understand. The newly-defined levels — remember, understand, apply, analyze, evaluate, 
and create — are expressed as verbs rather than nouns. This is appropriate to talk about 
thinking, which is, after all, an action. Also, as shown in The Taxonomy Table, the two 
highest levels now appear in reverse order from the original: create (synthesis) is now at 
level 6 and evaluate (evaluation) is now level 5. 



The Taxonomy Table 



The 


The Cognitive Process Dimension 


Knowledge 

Dimension 


1 

Remember 


2 

Understand 


3 

Apply 


4 

Analyze 


5 

Evaluate 


6 

Create 


A. 

Factual 

Knowledge 














B. 

Conceptual 

Knowledge 














C. 

P roced ural 
Knowledge 














D. 

Meta cognitive 
Knowledge 















Repro()LJce<l. pennasiof^. from Lor«) W. AnOeraort. and David R. KraUvwoM. A Taxonomy for Loamtng. 
JoacfunQ. anO A ssesstnQ. A Ravts»on of Btooma Taxonomy of £<SucaDonai Otofecuv^s. (Boston: ASyn and 
Bacon. 2001). p. 28. 
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As the Cognitive Proeess Dimension of the Taxonomy Table shows, another ehange 
ineorporates a eompletely new feature: Now the eognitive proeesses are aeeompanied by 
four dimensions of knowledge — faetual, eoneeptual, proeedural, and metaeognitive. 

When the cognitive processes are placed in a matrix with the dimensions of knowledge, 
teachers can classify instruction and assessment relative to both. For example, a question 
or test item that asks students to identify a species as vertebrate or invertebrate would fall 
on the matrix at cognitive level 2, Understand, and knowledge level B, Conceptual. A 
task that asks students to decide whether a vertebrate or an invertebrate could survive 
better in a particular environment moves up to cognitive level 4, Analyze, but stays at 
knowledge level B, Conceptual. Using a matrix such as this enables teachers to plan their 
assignments and assessments to take students beyond the levels of Remember and Factual 
Knowledge. 



I he C o»nitive Process Dimension 



Categories 


Alternative 


Definitions and 


& Cognitive 


Names 


Examples 


Processes 






1. Romombor — Retrieve relevant knowledge from long-term memory 


1.1 Recognizing 


Identifying 


Locating knowledge in long-term memory 
that is consistent with presented material 
(e.g.. Recognize the dates of important 
events in U.S. history) 


1.2 Recalling 


Retrieving 


Retrieving relevant knowledge from 
long-term memory (e.g.. Recall the dates of 
important events in U.S. history) 


2. Understaixl — Construct rr>eaning from instructional messages, including oral. 


written, and graphic communication 




2.1 Interpreting 


Clarifying 


Changing from one form of representation 




Paraphrasing 


(e.g.. numerical) to another (e.g.. verbal) 




Represen tNig 


(e.g.. Paraphrase important speeches and 




Translating 


documents) 


2.2 Exemplifying 


Illustrating 


Finding a specific example or illustration 




Instantia ting 


of a concept or principle (e.g.. Give 
examples of various artistic painting styles) 


2.3 Classifying 


Categorizing 


Determining that something belongs to a 




Subsuming 


category (e.g.. Classify observed or 
described cases of mental disorders) 


2.4 Summarizing 


Abstracting 


Abstracting a general theme or major 




Generalizing 


point(s) (e.g.. Whte a short summary of 
events portrayed on a videotape) 


2.5 Inferring 


Concluding 


Drawing a logical conclusion from presented 




Extrapolati ng 


information (e.g.. In learning a foreign 




Interp^ating 


language, infer grammatical princ^les from 




Predicting 


examples) 


2.6 Comparing 


Contrasting 


Detecting correspondences between two 




Mapping 


ideas, objects, and the like (e.g.. Compare 




Matching 


historical events to contemporary situations) 


2.7 Explaining 


Constructing 


Constructing a cause-and-effect model of a 
system (e.g .. Explain the causes of 






important 18th-century events in France) 


3. Apply— Carry out or use a procedure in a given situation 


3.1 Executing 


Carrying out 


Applying a procedure to a familiar task (e.g.. 
Divide one whole number by another whole 
number, both with multiple digits) 
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3.2 Impidmonting 


Using 


Applying a procedure to an unfamiliar task 
(e g., Use Newton's Second Law in situa- 
tions in which it is appropriate) 


4. Analyze— Break matenal into its constituent parts and determine how the parts 
relate to one another and to an overall structure or purpose 


4.1 Differentiating 


Discriminating 

Distinguishing 

Focusing 

Selecting 


Distinguishing relevant from inrelevant pads 
or important from unimportant parts of 
presented material (e.g., Distinguish be- 
tween relevant and irrelevant numbers in a 
mathematical word problem) 


4.2 Organizing 


Finding coherence 

Integrating 

Outlining 

Parsing 

Structuring 


Determine how elements fit or functbn 
within a structure (e g.. Stmcture evidence in 
a historical description into evidence for and 
against a particular historical explanatbn) 


4.3 Attributing 


Deconstructing 


Determine a point of view, a bias, values, or 
intent underlying presented material (e g., 
Determine the point of view of the author of 
an essay in terms of his or her political 
perspective) 


5. Evaluate — Make 


judgments based on cntena and standards 


5.1 Checking 


Coordinating 

Detecting 

Monitoring 

Testing 


Detecting inconsistencies or fallacies within 
a process or product: determining whether a 
process or product has external consistency: 
determining the effectiveness of a procedure 
as it is being implemented (e g.. Determine if 
a scientist's conclusions follow from the raw 
data) 


5.2 Critiquing 


Judging 


Determining inconsistencies between a 
product and external criteria: determining 
whether a product has external consistency: 
determining the appropriateness of a 
procedure for a given problem (e g.. Judge 
which of two methods is the best way to 
solve a given problem) 


6. Create— Put elements together to form 
elements into a new pattern or structun 


a coherent or functional whole: reorganize 

i 


6.1 Generating 


Hypothesizing 


Coming up with alternative hypotheses 
based on criteria (e.g.. Generate hy protheses 
to account for an observed phenomenon) 


6.2. Planning 


Designing 


Devising a procedure for accomplishing 
some task (e.g.. Plan a research paper on a 
given historical topic) 


6.3. Producing 


Constmcting 


Inventing a product (e g.. Build habitats for a 
specific purpose) 



Reproduced, penmasiort. from Lcnn W. Anderson and David R. KratvwoN. A Taxonomy for Learning. 

Teacfxng. and Assessing. A Revision of Bloom s Taxonomy of Educational Obfectives (Boston: AByn and 
Baoon. 2001). p. 31. 



A third feature of the new model is the addition of speeifie proeesses of thinking 
that oecur within each of the six cognitive levels. For example, level 4, Analyze, has 
these cognitive processes: differentiating, organizing, and attributing. In addition, the 
taxonomy gives definitions and examples. The Cognitive Process Dimension Talbe 
provides clues about ways to scaffold learning for students and to build assessments that 
require various levels of thinking skills. 
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Individual teachers can use the revised taxonomy, but its greatest potential for 
power may come with systematic, collaborative use. Teachers working in grade-level or 
subject- area teams could identify the key standards for a subject. Then, by standard, they 
could develop a bank of questions or tasks at each of the six levels. Although this would 
not be an overnight task, the time spent could result in a bank of quality assessment items 
at many different levels. Teachers would then have a tool for creating common 
assessments and for differentiating instruction and assessment for all students. 

When students become aware of the taxonomy, they also benefit. They learn a 
common language of learning and begin the invaluable process of metacognition — 
thinking about how they are learning. 

Using a taxonomy to enhance teaching and learning can be a powerful tool. Walsh 
and Sattes (2005) point out four important considerations to using this or another 
taxonomy. First, actual levels of cognition or dimensions of knowledge depend on the 
context of the question and the student’s knowledge and experience. Second, as Kathleen 
Cotton (1988) discovered, although teachers may pose higher order questions, the 
students’ responses do not match the cognitive level of the question approximately 50 
percent of the time. One way to attack this problem is to teach the taxonomy to the 
students. The third consideration is that most textbooks and teachers’ manuals contain 
questions at the lower end of Bloom’s Taxonomy. Teachers need to create their own test 
banks of higher order questions. Finally, there is an assumption that lower ability students 
cannot answer higher level questions. However, research tells us that all students can 
think at higher levels if they have adequate support and instruction (Bulgren, et ah, 2002). 
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A Primer of Common Assessment Terms 

Every educator needs to be familiar with common assessment. The following are some 
terms and their definitions: 

Reliability refers to consistency. A reliable test is one in which students obtain 
the same or very similar scores each time they take a test or an alternate form of the test 
proving that (1) relevant instruction or learning has NOT occurred between test 
administrations and (2) the student does not remember the questions or his/her responses 
from one test administration to the next. 

Validity means the test measures what it is supposed to measure. With a valid 
test, educators can draw reasonably accurate conclusions about student learning. 

Bias occurs in test items that offend or penalize a group of students. Tests can 
have gender, ethnic, disability, or other biases. The ideal test has no biased items. 

Norm-referenced tests measure students against a norm (average). In 
standardized assessments, large groups of students participate in a test administration and 
the average score for that group becomes the norm (50**' percentile). That is, half the 
population taking the test will be above the 50* percentile and half will be below. 

Criterion-referenced tests measure students against a set of standards, often 
called “performance standards.” Students are tested to determine whether they master 
specific standards. 

Constructed-response items on a test require students to create a response to a 
question. An example would be “Name three major reasons for the U.S. Civil War.” 

Selected-response items on a test require students to choose from a selection of 
possible answers. Examples are multiple-choice or true-false questions. 

Formative assessment occurs during instruction as a progress check. It is often 
used to evaluate the teaching and to give feedback to students. 

Summative assessment is given at the end of a lesson or a unit to determine what 
the student learned as a result of the instruction. 

Backward Design 

Begin with the end in mind. This adage, popularized by Steven Covey in his book 
Seven Habits of Highly Effective People, is at the heart of the principle of backward 
design advocated by Jay McTighe and Grant Wiggins. In their groundbreaking book. 
Understanding By Design, McTighe and Wiggins submit that “teachers are designers. An 
essential act of our profession is the design of curriculum and learning experiences to 
meet specified purposes” (1998, p. 7). Unfortunately, according to the handbook that 
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accompanies their work, teaehers often begin this design task by going to “textbooks, 
favored lessons, and time -honored aetivities rather than deriving them from targeted 
goals or standards” (1999, p. 37). 

Backward design involves teaehers in a three-stage proeess that begins with 
identifying desired results or outeomes. In the first stage, teaehers look to national, state, 
and distriet standards as well as to the needs and interests of their students in order to 
answer the following kinds of questions; What should our students know and be able to 
do at the end of this unit of study? What are the enduring understandings or essential 
questions that will drive eontent seleetion? 

Stage two in the proeess ealls for teachers to deeide what evidenee is required to 
demonstrate that students have indeed attained these outcomes. During this stage, 
teachers seleet and design assessments. They answer sueh questions as: How will we 
know if a student has the knowledge, skills, and understandings assoeiated with the 
desired results? How will we measure these student learnings? MeTighe and Wiggins 
(1998, 1999)suggest the use of a variety of assessment measures at different points in the 
learning task. Again, the Understanding by Design (UBD) framework deviates from the 
traditional approaeh in whieh teaehers typieally design assessments as a last task in the 
instruetional proeess. 

The eulminating stage of the UBD is the planning of learning experiences and 
instruetion. Here, teaehers answer the following kinds of questions: What enabling 
knowledge (i.e., facts, concepts, and prineiples) and skills (proeedures) will students need 
to perform effectively and achieve desired results? (1999, p. 40) What aetivities will 
engage learners in experienees that will lead to attainment of desired results? What 
materials and resourees will we need to provide? Aeeording to the authors, teaehers 
should be eoneemed with “uneoverage of eoneepts” rather than just eoverage of material 
during this stage. Uneoverage involves finding out students’ prior understandings of a 
partieular eoneept (e.g., gravity) and determining whether the student may be entering the 
unit of study with miseoneeptions. Direetly addressing miseonceptions is an important 
part of instruetion. 

Many sehools aeross the eountry are using UBD. The Assoeiation for Supervision 
and Currieulum Development (ASCD) offers a range of produets and serviees to support 
edueators who are adopting this approaeh. These are available on the ASCD Web site at 
www.ased.org. 
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A Six-Step Process for Using Student Data 

Because so many data are available these days, many teachers find them 
overwhelming and view assessment data as additional sets of records to file. Given the 
other demands on teachers’ time, we need to offer a simple and straightforward process 
for data analysis as we encourage the use of data. The following process, presented in 
Edvantia’s Data Connections CD-ROM, provides an easy-to-use template for individual 
and team use, and it can be a tool for school improvement specialists to use in their work. 

1. Determine your purposes for analyzing the data. Before you gather your 
data, prepare by thinking about what you want to learn from the scores. Do 
you have a school or district requirement you are addressing? Are you 
reviewing the effectiveness of your instruction? Or are you concerned about 
the performance of a student or group of students? Your goal is to derive 
information that will guide you in tailoring your curriculum and in meeting 
the specific needs of each student. 

For example, you may be analyzing the data to determine 

• how well your students performed at the end of the year 

• changes you might make to your instruction 

• how well classroom assessments predict performance on statewide 
assessments 

• an understanding of new students 

2. Develop a question list. After you know your purpose for analyzing the data 
(the “why”), you need to determine the specific questions you want to answer. 
What questions do the data raise about the curriculum, your students, and your 
teaching? For example, 

• In which areas did students perform well, and in which areas do they 
need more work? 

• Did my students who performed well in class also do well on state 
assessments? 

• How well are students doing over time? 

• How should I group students for differentiated instruction? 

You might be able to think of many questions. Focus on one or two of your 
most pressing questions because you can’t accomplish everything in one year. 
Think of this as an ongoing process. As you make improvements in your 
teaching effectiveness, you can look at new issues each year. 

3. Familiarize yourself with the data. Gather all the data you need. Consider a 
variety of classroom data, including gradebook, attendance records, past 
standardized tests, student portfolios, and qualitative data such as notes from 
parent conferences, observations of students, and so on. 
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4. Organize the data. Sometimes you’ll get data in the form of feedback sheets 
from the test publishers along with standardized test scores. These feedback 
sheets might be useful, or you may need to organize the data differently in 
order to answer your questions. 

5. Answer your questions. Using your judgment and professional expertise, can 
you answer the questions you raised in step 2? If you can’t answer the 
questions you raised, what data are you missing that could answer them? 
Sometimes you need to go beyond your gradebook and state report card. 

6. Develop action steps. The actions you take should relate directly to your 
questions. They may be pedagogical in nature or could relate to classroom 
management or test-taking strategies. Colleagues in your subject area are a 
good resource for addressing pedagogical concerns. If your students appear to 
have difficulty with a particular content area or set of standards, find out how 
other teachers in your subject address this content. Are they successful, or is 
there a trend that could be addressed by new activities, resources, or even 
professional development? 

The most critical aspect of your action steps is to develop steps that relate 
directly to your questions and the answers you discovered. If the data identify 
a problem with classroom assessments, your action steps should address 
classroom assessments. If the data identify attendance problems for an 
individual, your action steps should address solving those attendance 
problems. 



If your question is 


then you should 


In which areas did my students perform 
well/poorly? 


Look for group trends on standards. 


Did my students who performed well 
on classroom assessments also perform 
well on statewide assessments? 


Indicate test scores in your grade book 
and compare them with state data. 


How well are students doing over time? 


Collect data, such as state data, from 
the past three years. 


How should 1 group students as 1 
attempt to differentiate instruction? 


Note levels of performance on both 
classroom and state assessments and 
determine an effective grouping of 
students. 



As you identify problems, you also need to find solutions. Here are some 
problems and suggested solutions. 
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If you find 


then you should 


High absences correlate to poor 
performance. 


Communicate with parents. 


Whole class performance is poor on 
standardized test. 


Review and adjust curriculum and 
instructional strategies to better align 
with standards and test. 


Half of class did better than other half. 


Review class assignments. Examine 
how activities are structured. 


Performance is low across three years. 


Meet with prior teachers to align 
curriculum so that incoming students 
are better prepared. 



Data Connections 

Data Connections: Using Assessment to Improve Teaching and Learning is a 
multimedia CD-ROM that helps edueators better analyze, understand, and apply 
standardized test data in their elassrooms. The eourse also helps teaehers make 
eonneetions between standardized test data and other types of classroom assessment data. 
Most important, it assists learners in determining how to use these data to modify the 
curriculum and improve student achievement through a series of lessons on 

• test score utilization 

• curriculum alignment 

• classroom assessment 

• test-taking strategies 

For more information on Data Connections, visit www.edvantia.org. 

Book Review 

Accountability for Learning: How Teachers and School Leaders Can Take Charge by 
Douglas Reeves. Alexandria, VA: Association for Supervision and Curriculum 
Development, 2004. 

Douglas Reeves has spent years researching student assessment and teacher 
accountability. In his new book. Reeves takes a strong stance for student-centered 
accountability — looking at what students have learned as a result of their instruction. 
Using this approach. Reeves advocates a holistic evaluation system. 

Reeves identifies four components of teacher leadership in accountability required 
for students to reach their potential. The first component is observation of teaching 
processes and practices. Teachers must observe with unbiased clarity how often students 
resubmit work, how carefully teachers themselves follow best practices, how often and 
how well they collaborate with other teachers, and so on. As they observe, they begin the 
second component of leadership for accountability: reflecting on what worked, what 
didn’t, and what could be improved. The third component is synthesis, or putting together 
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conclusions from observations and reflections with information from available researeh. 
Finally, the fourth eomponent is eommitment to replieate praetiees that work. For 
example, Reeves asks, “We know that eollaborative seoring of student work is assoeiated 
with higher levels of fairness and greater levels of student performance. Will we expand 
it?” (p. 55). 

Reeves eonsiders the ehallenges presented by teaehing students from poverty, 
students from severely broken homes, students who do not speak our language, and a host 
of other issues. When he examines schools that have high student performanee. Reeves 
finds that performanee eannot be explained by differenees in student demographies or 
available funds. He sees improved student performanee as being tied to nine key 
professional praetiees of teaehers and leaders. Among these are several related to 
assessment: 

• strong eollaboration in examining student work, collectively defining 
exeellence, and building and collaboratively scoring common assessments 

• eommitment to frequent feedbaek to students in the form of eoaehing, 
praetice, and elear goals 

• sehools’ eonsideration of multiple sourees of student data and observation of 
data for groups of students over time 

• eommon assessments that are used eonsistently and seored eollaboratively 

Reeves eneourages the use of standards-based assessments rather than 
eomparative assessments. It is not enough to know that a group of students performed 
better than another group if neither group met a speeifie standard. Reeves differentiates 
between performanee indieators (e.g., adding double-digit numbers) and performanee 
standards (e.g., adding double-digit numbers with 95% aeeuraey). 

How do we reach the standard? Reeves is very elear: “Classroom assessment, 
ereated and seored by elassroom teaehers, is the gold standard in edueational 
aeeountability” (p. 1 14). Only through eonstant formative assessments and eollaborative 
eonversation about scoring can a faculty collect sufficient data to alter approaches, to 
integrate currieulum, and to assist students in meeting the standards. 

This is exeiting reading. In addition to offering the results of his researeh and 
challenging schools to replicate it. Reeves offers an appendix that outlines ways to 
develop holistie aeeountability from the sehool board office to the elassroom. 

Book Review 

Getting Excited About Data, Second Edition: Combining People, Passion, and Proof to 
Maximize Student Achievement by Edie Holeomb. Thousand Oaks, CA: Corwin Press, 
2004. 



This seeond edition of Edie Holcomb’s Getting Excited About Data focuses on 
how edueators ean eonneet the data at hand with their passion for teaehing and learning. 
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Acknowledging what the book is not — a comprehensive, bureaueratie, impersonal quiek- 
fix book — Holeomb proelaims that this edition is about the human element. 

From how to use data for alignment and aehievement to how to engage people, 
Holeomb suggests ways to involve educators in asking tough questions and finding 
answers for themselves. Of speeial interest are the many eharts and graphs that offer 
ideas for eollaborating, surveying for interest and knowledge levels of teaehers and 
students, and whole-sehool planning. The book, however, is not “touehy-feely.” There is 
plenty of hard data talk, just in user-friendly terms. 

Sehool improvement speeialists will appreeiate Holeomb ’s tried-and-true ideas 
for eonsulting with a faeulty using Carousel Data Analysis, Reporting Out, Round Robin 
Listing, Prioritizing, Data Day, and Go for the Green. In Chapter 1 1 : Drilling Down the 
Priority Data, she shares the how-tos for planning forward with student-speeifie data and 
planning baekward with skill-speoifie data. Step-by-step reporting of the author’s 
eonsulting ventures with reluctant-then-appreeiative partieipants hammers home her 
message that applying data to improve sehools is everybody’s business. 

Chapters 14 and 15 offer a detailed look at how to eonvert data and priorities to a 
school action plan; they lead and end with realistic ideas for “Sustaining the Struggle” 
and “Relentless Resilience.” In the author’s relaxed, first-person commentaries detailing 
what works in sehools (and sometimes what doesn’t), readers will find praetieal, 
insightful ideas that reveal Holeomb ’s zeal for eombining people, passion, and proof. 



A Portfolio Saved the Day 

The school improvement specialist stories that appear in Improving Sehools come from 
real life. The names have been changed or removed to preserve confidentiality. 

The young man, Fred, arrived as the school’s new Spanish teacher in the middle 
of the year. The previous teaeher had resigned after a long illness and many 
administrative problems. Fred’s direetions were to finish the year and teaeh as mueh 
Spanish as he eould in the remaining 14 weeks. Noneertified and eompletely unprepared 
for teaching adolescents, he eame to me, the school improvement specialist, in a panic. 
“They know nothing! I gave a quiz and they failed it miserably. Most of them have an F 
for the first semester’s grade. Their laek of voeabulary is amazing. How ean I ever teaeh 
them something in sueh a short time?” 

Fred and I talked at length and deeided to gamble on some strategies these 
students had never used before. Given the students’ apathy and history of failure, we 
deeided to give the students ehoiee and responsibility by having them build portfolios. 

As Fred taught his first unit, he had students grade their own and one another’s 
work. He kept student portfolios in his elass file eabinet, and students put their work in 
the portfolios. At the end of the unit, Fred gave these instruetions: “Choose your best 
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work from the areas of homework, quizzes, and translations. Take time to correct the 
work to make it the best you can. Then write an explanation of why this work is excellent 
and what you learned from it. Your grade will depend on the quality of the three pieces 
you turn in and your explanations.” With my help, he crafted a rubric to guide the written 
explanations. 

Students were amazed that they were only going to have to turn in three pieces for 
a grade. We heard comments like, “No fair! Why did 1 do all those other things if he 
wasn’t going to use them?” We also heard, “Gosh, you mean we can make it better — 
won’t that mean a better grade for us?” 

Fred and 1 held our breath, but the students actually went to work on the task. 

Even students who had spent much time with their heads down on their desks came to life 
and tried to resurrect some assignments. The results were more than credible. The three 
pieces the students turned in showed students that they were capable of producing quality 
work. The companion reflections made the students consider what they were learning and 
how they learned it. 

We tried another unit. Then another. Student learning improved gradually, with 
some students making remarkable progress. Fred decided he needed to collect what he 
now called “samples of excellence” more frequently than at the end of a unit. He got 
feedback from the students on refining the rubric. When he became discouraged at the 
quality of some of the work, he learned to give more powerful assignments. At the end of 
the year, Fred asked the students to prepare their portfolios to present to their Spanish 
teacher next year. Fred’s students spent their time perfecting recent work, writing 
messages to the new teacher about their learning, and making suggestions for next year’s 
class. Some students went all out with artwork and audiotapes. 

Did these students who had been deprived of good Spanish instruction for most of 
the year make up their deficits? Fred and 1 don’t think so. But we do know that the 
practice of peer and self-assessment, combined with reflection and the opportunity to 
perfect a few assignments, made a big difference. Fred created motivation and 
opportunity to learn. The student portfolios gave students confidence that they could 
learn. 

Student Self-Assessment Instruments 

. Weekly learning logs 

• Reflective responses (The most interesting thing 1 learned today was . The 

thing 1 still have a question about is .) 

. Self-assessment checklists 

. Rubrics 

• Teacher-student interview 

• Response to this question: The reason this work is high quality is . 

. “Traffic light” icons to label a level of understanding: good, a little, or no 
understanding. 
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Book Review 

Using DATA to Improve Student Achievement: A Handbook for Collecting, Organizing, 
Analyzing and Using Data by Deborah Wahlstrom. Suffolk, VA: Successline, Inc., 2002. 

Deborah Wahlstrom’s book opens with a rationale for data collection, then moves 
into user-friendly types and sources of data in three categories: outcome, demographic, 
and process. She presents a four-step process for looking at data: collect, organize, 
analyze, and use. 

Chapter 3 defines and illustrates terms for “talking the talk” of data. Chapters 4 
and 5 “walk the talk” of data disaggregation and the construction of useful tables. 
Throughout these chapters are practical, relevant ways of looking at data, presented in 
readable, informative charts, graphs, and definitions. 

How-to sections (Chapters 6 and 7) lead readers into creating visual tables and 
pose questions for “thinking deep.” From discussions about criterion-referenced 
questions to attendance and various forms of assessment, Wahlstrom provides factors to 
analyze for a comprehensive school study. Quick tips, ready-to-use templates, and data 
organizers invite hands-on participation. 

The final three chapters put Wahlstrom’s philosophies to work as readers find 
uses for data in curriculum alignment, student-centered instruction, and whole-school 
planning. Everything from charting to implementing and monitoring is discussed in easy- 
to-read text supported with illustrations. This book could become one of the most-used 
resources for faculty collaboration. 



Rubrics For Assessment 

A class has a project due in two weeks. The students ask, “What counts? What do 
we need in the project? How will I know if my project is good enough?” The teacher 
receives a set of projects and wonders, “How will I assign a grade to these? What factors 
are really important? If one has clever artwork and another has no artwork, does that 
make a difference?” 

Thoughtfully creating a rubric prior to making the assignment can make life much 
easier for the students and the teacher. Spence Rogers tells us that a rubric clarifies “what 
counts” in an assignment. Most rubrics pick the key criteria of an assignment and detail 
the different levels of quality for those aspects. Look at the example below. 
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Rubric for History Timeline 




Outstanding: 4 


Very Good: 3 


Could 
Improve: 2 


Needs Help: 1 


Aecuraey of 
Information 
(80%) 


All required 
information is 
provided, with 
additional 
clarifying or 
interesting 
information. 
Events appear 
in the eorrect 
sequence. 


All required 
information is 
provided. 
Events appear 
in the eorrect 
sequence. 


Some 

information is 
missing. Some 
events are 
missing or not 
in the correet 
sequence. 


A signifieant 
amount of 
information is 
missing and/or 
not in eorrect 
sequence. 
Timeline is 
difficult to read 
because of the 
errors. 


Presentation 

Qualities 

(10%) 


Easy to read 
with neat 
lettering. There 
are no erasures 
or illegible 
writing. The 
timeline has 
additional 
features, sueh 
as pictures or 
graphics. 


Easy to read 
with neat 
lettering. 
Writing is large 
enough to read 
and evenly 
spaced. There 
are no erasures 
or illegible 
writing. 


The timeline 
ean be read and 
understood. 

The events may 
not be evenly 
spaced and/or 
writing may 
not be large 
enough. May 
contain some 
erasures or 
illegible 
writing. 


The timeline has 
erasures and/or 
illegible writing 
that interfere with 
readability and 
understandability. 
Spaces and size 
of writing are 
erratic/uneven. 


Meehanics 

(10%) 


No errors in 
capitalization, 
punctuation, or 
spelling. 


Only minor 
errors in 
capitalization, 
punctuation, or 
spelling. 


A few errors in 
capitalization, 
punctuation, or 
spelling. 


Signifieant errors 
in capitalization, 
punctuation, or 
spelling. 



In this case, the teaeher wants accurate information and also a clean and clear 
presentation and proper meehanics. If you were a student, wouldn’t you know how to 
ereate your timeline? And wouldn’t the teaeher find it easy to assess the quality of your 
work? In addition to identifying and deseribing the key eriteria for the timeline, this 
rubric spells out how mueh eaeh category eounts toward the grade. 

General Types 

Judith Arters defines two general types of rubrics that have different uses and 
values. A holistic rubric provides a single score based on an overall impression of the 
student’s work. Holistic rubrics are good for getting a “snapshot” of the quality of work. 
They are also fairly quick to grade, making them excellent choices for large-seale 
assessments, sueh as state writing assessments. Holistic rubrics are fine forjudging work 
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that has only one trait to score or forjudging the overall impact or success of a work. For 
example, how well did the student persuade the reader? 

Analytic trait rubrics divide the task or assignment into areas or traits that can be 
judged separately. An example might be a writing rubric that contains separate gradations 
for organization, voice, word choice, and mechanics. Analytic trait rubrics are good for 
judging complex assignments. These rubrics provide specific information and feedback 
to students and help the teacher identify individual and class strengths and weaknesses 
for instructional planning. Analytic trait rubrics are excellent for peer assessment or for 
coaching students to meet a higher standard. Because they can provide formative 
feedback to students, these rubrics are most often the choice for general classroom use. 

Tips for Creating Sound Rubrics 

• Decide on the key criteria for the assignment and create levels of quality for 
each. 

• A 4-point rubric is comfortable for classroom use. There is always the implied 
“zero,” which means no work was turned in. 

• For a 4-point rubric, define “3” really well. A score of “3” meets the standard; 
anything better is a “4.” Thereafter, it is fairly easy to define a “2” and a “1”. 

• Have levels build on one another; for instance, a “3” has everything a “2” has 
and more (more information or fewer errors). 

• When possible, aim for a generic rubric that can be used for several 
assignments. The rubric can be modified and improved over time, and 
students can become familiar with it. 

• Look for examples of quality work. Use the examples to build your rubric and 
to share with students. 

• Post the rubric in the classroom and give copies to students and parents. 
Rubrics should not be secret. 

(Adapted from The High Performance Toolbox by Spence Rogers and Shari Graham.) 

Student Involvement 

Involve students in creating a rubric. Spence Rogers and Judith Arters suggest 
that teachers provide exemplars — examples of excellent work. Students then form groups 
to brainstorm the qualities common to the examples. Those commonalities and their 
descriptions become the beginnings of a rubric. Building a classroom rubric is a powerful 
learning experience because the group must analyze and compare the exemplars, pull out 
key elements, and create definitions. Using these higher order thinking skills gives 
students insights into completing their own tasks and builds ownership and self- 
confidence. (Rogers & Graham [1998]; Arters & McTighe [2001]). 
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When to Use a Rubric 

When is it important to have a rubrie? Not every assignment needs a rubrie, but 
experts say a class should have a rubric, even a very simple one, for all constructed- 
response assessments. 

Arters and Rogers suggest using rubrics, especially the analytic trait variety, to 
coach students toward meeting a specified performance goal. If the purpose is to assess 
for learning by giving students specific feedback, the rubric will do that and grades can 
be assigned through logic or equivalencies (Rogers & Graham, [1988] p. 201; Arters & 
McTighe, [2001] p. 81). 

Online Resources 

http://rubistar,4teachers,org/index,php 

http://myt41,com/index,php 

MyT4L and RubiStar are sites that offer templates for creating rubrics, plus 
examples. 

http://www,pz,harvard,edu/Research/RubricSelf,htm 

Harvard’s Project Zero presents research on the effects of using rubrics in the 
classroom. 

http://school.discovery.com/schrockguide/assess,html#web 

Kathy Schrock’s Guide for Educators offers instructions, sample rubrics, links to 
other resources, and more. 

http://edtech,kennesaw,edu/intech/rubrics,htm 

The Georgia Education Technology Training Center at Kennesaw State 
University presents information on making and using rubrics. 
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The Dreaded Algebra Test: How Some Teachers Attacked It 

The school improvement specialist stories that appear in Improving Schools come from 
real life. The names have been changed or removed to preserve confidentiality. 

Teachers in a large urban high sehool where I was the sehool improvement 
speeialist remained in a constant state of worry and coneern for their Algebra I students. 
These students needed to pass a state examination to graduate from high school, and 
many of them laeked even rudimentary math skills. As they eonsidered strategy after 
strategy, teachers read Mike Sehmoker’s work (1999), whieh advocates a elear and 
focused goal, collaborative teamwork, and consistent use of student data. They read 
literature from Douglas Reeves (2004), Riek DuFour (2003), and others who advoeate 
developing and administering eommon benehmark assessments, then scoring them 
collaboratively. 

The teachers deeided to give common assessments a try. To start, the teaehers sat 
down and eharted a pace for teaching each of the standards during the year. They deeided 
to give a test every Friday to assess mastery of the standards they taught that week. As 
they graded the tests on Friday afternoons, they soon realized that the students were not 
retaining knowledge of the algebraie eoneepts from week to week. What eould be the 
answer? With students who came to school without the prerequisite baekground 
knowledge, the teaehers knew they needed repetition and eonstant review of the key 
principles. 

To inelude an intensive review of all eoneepts eaeh Friday seemed an impossible 
task: the quiz would quiekly turn into a two-hour test. Instead, the teaehers deeided to 
review all the previous concepts and indieators during the first 10 minutes of every elass. 
Over the course of the year, the teaehers developed a elass “starter” aetivity for eaeh day. 
This activity consisted of 10 math problems to be completed in 10 minutes. Eaeh of the 
10 questions reviewed a key algebraic concept. Every week, students completed 50 
review problems and graded their own work with the teaeher’s assistance. As new 
indieators were taught, they gained a plaee in the starter aetivities. When the teaehers 
diseovered eoneepts the students were really struggling with, those eoneepts earned more 
places on the starter activities. 

Meanwhile the teachers eontinued their Eriday collaborative assessments. 
Completing these tests and grading them together enabled the teaehers to diseuss their 
teaehing methods, to talk about students’ key needs, to modify their pacing charts, and, of 
course, to modify their starter aetivities. 

These Algebra I teaehers had a elear focus, strong teamwork, and used consistent 
data to provide a strong math program to their students. The first year of their 
eollaborative efforts yielded impressive results. By their third year of eollaborative 
problem solving and grading assessments, the students in their high sehool had reached a 
93% passing rate on the state exit examination. Both teachers and students reached an 
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amazing goal. They demonstrated the power of a elear foeus in teaehing that was baeked 

up by using and seoring eommon assessments. 
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Zeros for Missing Work? 

Douglas Reeves, in his online newsletter from the Center for Performanee 
Assessment (http://www.makingstandardswork.eom), offers an interesting opinion about 
giving a zero when a student does not turn in an assignment. He points out that grades 
refleet relatively equal intervals. For example, an A is 90-100, a B is 80-89, a C is 70-79, 
and a D is 60-69. If so, wouldn’t an F be 50-59? Reeves points out that a grade of zero is 
about six times as bad as an F! 

Think about what a zero does to a student’s grade. Suppose Charlie is a borderline 
student with 10 grades that average 71. If he fails to turn in his next assignment and gets 
a zero, his average falls to 64. If he gets two zeros, the average falls to 59. Three zeros 
drop the grade to 53. It’s easy to see that a eouple of zeros ean devastate a borderline 
student’s average. 

Yes, says Reeves, we need to emphasize timeliness and responsibility, but we also 
need to teaeh the Charlies in our elasses. Beeause many students are not bothered by a 
low grade or a zero, wouldn’t a better “punishment” be a requirement to eomplete the 
work? If students who do not turn in work must eomplete it by forfeiting time from 
luneh, ball praetiee, homeroom, or the like, they may soon learn to eomplete work on 
time. 



Reeves advoeates giving a grade of 50 for work that is never turned in, so the 
student ean ultimately retrieve his grade average. If Charlie reeeived three grades of 50 
for his missing work, his new average would be 66. He’s still failing the elass, but his 
ehanees of bringing the semester average to passing have just inereased dramatieally. If 
the teaeher assigns Charlie additional work to eompensate for the missed work, the 
teaeher has the opportunity to give work that may be more appropriate to his 
developmental level than the original, missing assignment. 

Think about it. Giving a zero is often a punishment rather than an attempt to 
provide a student with feedbaek. Doesn’t teaehing mean offering better alternatives? 
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When Planning Is Not Enough 

The school improvement specialist stories that appear in Improving Schools come from 
real life. The names have been changed or removed to preserve confidentiality. 

It was my second year as a sehool improvement speeialist at a middle sehool that 
had never made adequate yearly progress. The sehool leaders and I were not totally 
diseouraged, but a number of school improvement ideas had not taken hold as we 
expeeted. 

When the prineipal returned from a fall retreat, she greeted the leadership team 
with enthusiasm: “We need to do some formative assessments!” (I smiled approvingly.) 
“You know,” she said, “we purchased formative assessments from a company last year, 
but the questions didn’t really mateh the standards we taught. All of us need to get 
together and write our own, and then see how the students perform.” (I smiled again, 
stopping short of a standing ovation and a big shout-out. After all, a good idea from any 
souree is a good idea.) 

So the prineipal wrote an e-mail to the staff: “If you are a teaeher of a eore 
aeademie subjeet, please turn in four multiple-ehoiee questions that eover the standards 
you have taught to date.” She assigned a due date one week henee, a drop-off loeation, 
and a teaeher to eoordinate and publish all questions. At the end of the week, however, 
not a single question had been submitted, and we were within three weeks of doing a 
whole-school formative assessment. My eonversations with teachers made it elear that 
this was a principal directive with no explanation, no support. They just didn’t know what 
was expeeted of them. 

I asked the principal if I could facilitate a session on how to write quality 
questions aligned with standards at the next grade-level meetings. She liked the idea, so I 
worked up some aetivities and led teaehers in discussion. They assured me they 
understood what to do. At the end of another week, we still had no questions for the 
formative assessment. 

After a frank diseussion with the prineipal, we both agreed that teaehers still were 
unelear about how to construet good questions, hadn’t bought in to the idea of formative 
assessments, or had little experienee in eollaboration. For the next grade-level meetings, 
we deeided on a make-and-take format. 

Teachers arrived with paeing guides, texts, and workbooks. We provided sample 
questions, paper, and peneils. The teaehers looked at the standards they had taught, 
worked in groups, and produeed questions in the same format as the state test. Almost 
without fail, teaehers thanked us for the opportunity to work together. The prineipal and I 
were gratified by both their produetivity and the well-erafted questions they produeed. 

When the prineipal and I refleeted on the experienee, we agreed that planning 
without elear explanations doesn’t always work. The right atmosphere, appropriate 
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materials, and enough time for eollaboration — along with a lot of team work and a little 
“gentle persuasion” — ^took us a long way toward aoeomplishing our goal. 

Contributors to this issue of Improving Schools include school improvement specialists 
Rusha Sams and Susan Hudson and Appalachia Educational Laboratory at Edvantia staff 
members Jackie A. Walsh and Nancy Balow. 
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Introduction 



The accountability demanded by the No Child Left Behind (NCLB) Act of 2001 has 
focused educators’ attention on assessment at multiple levels (grade, course, school, and 
district) and on the results of various groups of students within the overall student population 
(racial and ethnic groups, economically disadvantaged, students with disabilities, and limited 
English proficient). This heightened awareness has created a need for classroom assessment 
strategies to ensure the success of all students. 



Review of the Literature 

Assessing or evaluating student performance is not new. Tyler (1966; cited in Jones, 
2001) describes three purposes of assessment: (1) to assign grades or classify students, (2) to 
plan for subsequent teaching, and (3) to evaluate the effectiveness of a change in curriculum 
or methods. 

Black and Wiliam (1998a), however, write about only two functions of assessment — 
formative and summative. Summative assessments are very much a part of teachers’ work. 
Teachers use the results of summative assessments to report to parents through periodic 
report card grades and/or teacher-parent conferences, and these assessments determine 
whether students pass the course and are promoted to the next grade. Summative assessments 
can also serve purposes external to the classroom. According to Black and Wiliam, “Such 
assessment becomes formative assessment when the evidence is actually used to adapt the 
teaching to meet student needs” (pp. 1-2). 

The use of formative assessment reflects the behaviorist (stimulus-response) theory 
articulated by Skinner (1954). Building on behaviorist theory. Bloom (1976) proposed a 
theory of schooling formulated on three constructs: (1) student characteristics, (2) quality of 
instruction, and (3) learning outcomes. Student characteristics include cognitive entry 
behaviors and affective entry characteristics. Quality of instruction, incorporated in learning 
tasks, includes the extent to which the cues, practice, and reinforcement of the learning are 
appropriate to the needs of the learner. Three learning outcomes are defined: the level and 
type of achievement, the rate of learning, and affective outcomes. 

According to Bloom, when student entry characteristics are favorable and the quality 
of instruction is optimal, learning outcomes will be high. Further, it is possible to modify 
both students’ entry characteristics and the quality of instruction so that learning outcomes 
are high for 90 to 95% of the students. 

Bloom set out to confirm his theory through a series of experiments. Testing students 
after each learning task. Bloom (1976) found that, if students who have not met the criteria 
for mastery are then presented with instructional content in a different way, they will not only 
possess the affective disposition for the next learning task but will also have the cognitive 
skills to learn the next task (mastery conditions). However, if students receive no feedback or 
instruction after an assessment, their performance continues to decrease on subsequent tasks 




and their affective disposition toward learning also declines (nonmastery conditions). 

Further, the achievement gap between mastery and nonmastery students widens as 
instructional tasks continue to be given. This is because mastery students receive feedback 
and instruction, while nonmastery students do not. Students who receive such mastery 
treatment over three or more learning tasks score higher than the nonmastery students who 
receive no feedback after initial testing for each of the learning tasks. 

Moreover, Bloom’s research indicates that if students who initially did not master a 
task are provided with feedback and additional instruction, they quickly score better than the 
group of students provided with no feedback or additional instruction. This experiment was 
repeated using the same design in a variety of subjects: second language, imaginary science, 
matrix algebra, and elementary probability. 

Bloom suggests that achievement within each learning task is attainable for all 
students if feedback and additional instruction are provided. This information counters claims 
that lack of achievement is a product of factors over which schools have no control (such as 
socioeconomic status). Bloom also posits other variables and shows their association with 
learning outcomes. These variables include “cues, participation of the learner in the learning 
activity, and reinforcement which the learner secures in relation to the learning” (1976, p. 
115). 



Based on his research. Bloom (1976) concludes that feedback and corrective 
procedures account for 25% of the variation in student learning. Although Bloom focuses on 
individual students instead of classrooms, schools, or school systems, his theory suggests that 
schools and classroom teachers who provide consistent feedback on instruction to lower 
performers could achieve better results. 

Implemented in schools as mastery learning or outcomes-based education. Bloom’s 
approach has been studied in many different settings. In all. Block, Efthim, and Burns (1989) 
reviewed 48 studies and conclude: 

Mastery learning approaches seem to work comparatively well almost all 
of the time. That is, they typically produce effects that are greater than or 
equal to a non-mastery approach. . . . Mastery learning approaches have 
comparatively strong effects on general student achievement. Assuming 
normality of scores, a median effect size of .76 [from the 48 studies] means 
that the typical application should move the average 50**^ percentile student 
to about the 77**' percentile in achievement, (p. 28) 

Nearly 10 years later. Black and Wiliam (1998a) reported the results of an extensive 
survey of the research literature on formative assessment. Their criteria for inclusion in the 
review were that “quantitative evidence of learning gains was obtained for those involved 
and for a similar group not so involved” (p. 2). Out of 580 journal articles and book chapters, 
they found 250 that met their criteria and were also responsive to this question: Is there 
evidence that improving formative assessment raises standards? 




In another analysis, Black and Wiliam (1998b) selected 23 studies from a research 
review published by Fuchs and Fuchs (1986) that concentrated on classroom assessment for 
children with mild handicaps and 20 more studies conducted after the review by Fuchs and 
Fuchs. All together, the studies investigated kindergarten through undergraduate students, 
several content areas, and several countries. 

Studies examined by Black and Wiliam (1998b) showed that strengthening the 
practice of formative assessment produced significant and often substantial gains. The 
learning gains were measured using effect size, which is calculated by comparing the average 
improvement in the test scores of pupils involved in a treatment with the range of scores that 
are found for typical groups of pupils on the same test. The ratio of the former divided by the 
latter is known as the effect size. Typical effect sizes found in the formative assessment 
experiments were between 0.4 and 0.7. Another important finding was that formative 
assessment helps low achievers more than other students and so reduces the range of 
achievement (achievement gap) while raising achievement overall. 

More recently, Wiliam, Lee, Harrison, and Black (2004) conducted a study of 
secondary school students whose teachers were asked to dedicate time to formative 
assessment and instruction. Twenty-four teachers (two math and two science teachers from 
each of six schools located in two school districts in England) incorporated formative 
assessment in their classrooms. Classes of teachers who participated in the study were 
matched with equivalent classes, and student performance was compared. Effect sizes were 
calculated for each pair of classrooms. The differences in achievement between the treatment 
groups and the comparison groups resulted in a median effect size of .27 and a mean effect 
size of .34. This study is noteworthy because the intervention was conducted with teachers, 
so the effects on students were indirect. 

Guterman (2002) examined the use of metacognitive awareness guidance to provide 
feedback on formative assessments to see if such guidance is associated with a difference in 
the learner’s level of performance and achievement and also the learner’s chances of 
internalizing the guidance component. Guterman describes the use of self-talk, a process of 
developing instruction material that helps students and teachers perform activities. Guterman 
observed 300 students from four schools in Israel (two schools in upper-class neighborhoods 
and two schools from lower-middle-class neighborhoods). Students were randomly assigned 
by classrooms to one of three groups: control, placebo, and experimental. Three different 
versions of reading assessment tasks were produced: 

• The control group was asked to read specific text and answer a set of questions 
that followed. 

• In the experimental group, written metacognitive guidance preceded reading of 
the passage and helped students engage in some of the larger issues suggested in 
the passage. 

• The placebo group was asked to read the content instruction and the text only. 
When students finished reading, they were asked to raise their hands to signal that 
they were ready to proceed with the reading assessment tasks. They were then 
given special instructions on how to proceed, similar to the experimental group. 




All participants were given a standardized reading comprehension test developed by the 
Israeli Ministry of Education; grades in reading and writing were obtained from school files. 
A one-way analysis of variance performed on the reading ability scores demonstrated that 
random assignment resulted in three groups that prior to the intervention were essentially 
equal in reading ability. All students were also given the Metacognitive Strategy Index, a 
questionnaire that measured their awareness of metacognitive reading strategies. All reading 
assessments were graded in a standard way. The analysis demonstrated that following the 
intervention, “learners who were given written metacognitive awareness guidance built on 
their prior knowledge exhibited higher aehievement on the task” (Guterman, 2002, p. 296). 

Stecker and Fuchs (2000) used a Curriculum-Based Measurement (CBM) system to 
provide individual feedback on the results of curriculum-embedded assessments to a group of 
42 special education students in Grades 2 through 8. Each of the 42 students in the 
experimental group had a partner, and the two students received the same feedback rather 
than feedback based on their own results. Stecker and Fuchs found that “students whose 
teachers tailored instructional adjustment based on those students’ own CBM data performed 
significantly better on a global achievement test than did their partners whose instructional 
adjustments were not based on their own assessment data” (p. 128). The authors suggest that 
the study shows the importance of making instructional adjustments with the child’s 
assessment results in mind. 

Rea-Dickins (2001) conducted a case study of assessment use by two language- 
support teachers and one mainstream class teacher in inner-city schools that had a high 
density of English language learners (up to 98%). Data collected included classroom 
observations and interviews with classroom teachers and language support teams. 

Rea-Diekins identified three different charaeteristics of classroom-based assessment: 
bureaueratic, pedagogic, and learning. The bureaueratic demands of assessment serve the 
obligation that schools must provide feedback to parents and external agencies. The 
pedagogic phase refers to instructional decisions based on group and individual learners’ 
performance. The learning phase is embedded within instruction. Rather than measuring 
learning, assessment helps the learner monitor his or her own learning. This study 
demonstrates that classroom assessment practice is related to determining what the learner 
knows and the amount of progress made. This determination then helps to guide teachers’ 
decisions about what and how to teach the next lesson. 

Brookhart and Durkin (2003) collected data from 96 students in 12 social studies 
classrooms in an urban high school, including pre- and post-survey results, anecdotal records 
from students’ assessments, interviews, and classroom observations. In addition, the 
researchers interviewed two students per class, one low achiever and one high achiever. 
Brookhart and Durkin found that classroom assessment appears to be associated with three 
types of student motivation: (1) wanting to learn for learning’s sake, (2) wanting to show 
what they learned, and (3) wanting to help others learn or to learn from others. The 
researchers conclude that students want to learn more when the content of the assessment is 
connected with “a conscious, expressed interest” (p. 43). 




While experimental researeh indieates that appropriately designed and implemented 
mastery learning models and other types of formative assessment ean inerease student 
aehievement, formative assessment has proven diffieult to implement in praetiee beeause 
students are no longer progressing in unison (Jones & Spady, 1985). The use of eomputer- 
based formative assessment may hold promise for overeoming student instruetional grouping 
and seheduling diffieulties. Ediger (2000) asserts that beeause eomputer paekages involve 
text with read, respond, and eheek sequenees, students take small steps in responding to 
multiple-ehoiee test items that eover the eontent of the assignment and get instant feedbaek 
without teaeher involvement. The eomputer software program keeps traek of the eorreet 
responses and diagnoses errors within eaeh sequential step of learning so the teaeher also 
reeeives immediate feedbaek from the software program. 

Chudowsky & Pellegrino (2003) suggest that the use of teehnology may help remove 
some of the eonstraints assoeiated with eurrent large-seale assessment praetiee. Computer- 
based assessments have the eapability to assess problem-solving skills and also to identity 
the sequenee of aetions taken by the learner in solving the problem. Teehnology holds the 
promise for ereating assessments that indieate how students think and reason while they are 
engaged in important learning aetivities. This information provides teaehers with a profile of 
student learning. In addition, eomputer-based assessment information ean be eolleeted during 
the normal instruetional proeess rather than interrupting the learning proeess to administer 
external tests. 

In order for teehnology to beeome an effeetive tool for assessment, many issues need 
to be addressed. For example, MeFarlane, Williams, and Bonnet (2000) point out that a 
framework for assessing the quality of the eontent of the teehnologieal tools and the 
proeedure for learning needs to be established. Media genres, learner progression, and 
teaehing goals and approaehes must also be addressed. There should be a eommon 
terminology for addressing multimedia. If used properly, teehnology is powerful as a 
diagnostie tool. It ean be used to support judgments about student performanee and to 
eonduet self-assessment of strengths and needs. To get an up-to-date overview of the issue, 
see reports from lAETE national eonferenees on teehnology for assessment at 
http://www. edvantia.org/eorporate/indexI. efm?&seetion=about&area=iaete. 

Peat and Franklin (2002) surveyed 700 first-year biology students at the University of 
Sydney to determine students’ pereeptions of eomputer-delivered quizzes. A shortage of 
qualified faeulty and inereased enrollment at the university ereated a need to find new ways 
to improve student learning. One strategy was eomputer-based learning. Students took 
weekly quizzes, moek examinations, and speeial self-assessment modules using the 
eomputer. The students who used the eomputer program reported that it helped them to sort 
out details in eontent areas where neither the textbook nor the professor were very useful. 
Most of them had used the eomputer as a learning tool and self-assessment before the final 
exam. Peat and Franklin found evidenee that student learning outeomes improved with the 
use of eomputer-based assessments. This inerease in aehievement appears to be assoeiated 
with improved student attitudes toward learning. Students who used the eomputer program 
reported that they were eneouraged by the instant feedbaek reeeived. While the preliminary 




findings are promising, there is a need for experimental studies to test the effectiveness of 
computer-based assessment. 



Summary 

Classroom assessment is used for many purposes. A substantial amount of 
scientifically based research has shown that classroom assessment can help to improve 
student achievement if the results are used to plan subsequent instructional activities. 
Students who don’t initially learn a concept will need feedback, more time, and additional 
instruction. If these students do not receive appropriate feedback and additional instruction, 
they will fall further and further behind. As promising as the use of formative classroom 
assessment is, it is difficult for teachers to manage the differences in individual student 
progress. Computer- assisted instruction and assessment show promise for helping classroom 
teachers overcome these management issues. 
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